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ABSTRACT 


Existing Web browsers handle security errors in a manner that of- 
ten confuses users. In particular, when a user visits a secure site 
whose certificate the browser cannot verify, the browser typically 
allows the user to view and install the certificate and connect to 
the site despite the verification failure. However, few users under- 
stand the risk of man-in-the-middle attacks and the principles be- 
hind certificate-based authentication. We propose context-sensitive 
certificate verification (CSCV), whereby the browser interrogates 
the user about the context in which a certificate verification error 
occurs. Considering the context, the browser then guides the user 
in handling and possibly overcoming the security error. We also 
propose specific password warnings (SPW) when users are about 
to send passwords in a form vulnerable to eavesdropping. We per- 
formed user studies to evaluate CSCV and SPW. Our results sug- 
gest that CSCV and SPW can greatly improve Web browsing se- 
curity and are easy to use even without training. Moreover, CSCV 
had greater impact than did staged security training. 


Categories and Subject Descriptors 


H.5.2 [Information Systems]: User Interfaces— interaction styles, 
screen design, evaluation; H.1.2 [Information Systems]: User/- 
Machine Systems—human factors; 1.3.6 [Computing Methodolo- 
gies]: Methodologies and Techniques— interaction techniques 


General Terms 


Security, Human Factors 


Keywords 


Web browser, HTTPS, SSL, certificate, password, man-in-the-mid- 
dle attack, eavesdropping attack, just-in-time instruction, well-in- 
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1. INTRODUCTION 


Users often find the Web’s hypertext and browsing paradigms 
attractive, intuitive, and easy-to-use. Consequently, Web-based in- 
terfaces are now used in a wide variety of client-server applications. 
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Among those applications, there are many that are security- 
sensitive, including access to bank and other financial accounts, 
e-commerce sites, and Web-based email. In such applications, se- 
curity breaches could cause users major financial or privacy losses. 
However, many users employ Web browsers in such applications 
without ever receiving formal training on how to do so. 

The technology necessary for securing Web applications is gen- 
erally considered to be well-understood. Secure Web servers typi- 
cally use the HTTPS protocol (34). HTTPS layers HTTP (Hyper- 
text Transfer Protocol) p) over SSL (Secure Socket Layer) (or 
its standard equivalent, TLS (Transport Layer Security) (7). SSL 
offers communication confidentiality and integrity with certificate- 
based server authentication and protection against replay attacks. 
Using up-to-date cryptographic algorithms, such as SHA-1 
and AES {17}. SSL can provide a high level of security. 

However, the usability of this security technology has received 
surprisingly little attention in the literature (27). User interfaces 
that are difficult to learn or prone to misuse can expose users to 
unacceptable risks, even if the underlying technology is secure 
A 

This paper addresses three questions on the usability of Web 
browser security. First, given a group of computer-literate users, 
how likely is it that an attack against them will succeed, in repre- 
sentative security-sensitive Web applications? Second, is it possi- 
ble to foolproof Web browsers, such that they can be used securely 
even by untrained computer-literate users? Third, can education 
about the relevant security principles, attacks, and tools improve 
the security of how users browse the Web? We consider specifi- 
cally attacks enabled by tools that are easily available on the Web: 
man-in-the-middle (MITM) attacks against HTTPS-based access to 
financial accounts and e-commerce sites |28], and eavesdropping 
attacks against HTTP-based email access [35] [8]. Given the ease 
with which such attacks can be deployed even by inexperienced 
attackers, the probability that they succeed should be very low. 

We performed a user study to answer the first question. Our 
results show that, with current users and Web browsers, the men- 
tioned attacks are alarmingly likely to succeed. More often than 
not, users’ behavior defeats the existing Web security mechanisms. 

In response to the second question, we contribute two novel user 
interface techniques for Web browsers, CSCV (Context-Sensitive 
Certificate Verification) and SPW (Specific Password Warnings). 
CSCV’s goal is to thwart MITM attacks against HTTPS and other 
protocols that use certificates to authenticate servers. SPW cautions 
users against sending passwords in a form vulnerable to eavesdrop- 
ping. We implemented CSCV and SPW in a Web browser and 
evaluated them in a second user study, involving the same users 


and attacks as the first study. CSCV blocked MITM attacks against 
HTTPS-based applications completely. SPW greatly reduced the 
insecure transmission of passwords in an HTTP-based application. 
Although untrained, users had little trouble using CSCV and SPW. 
These results suggest that, at least for some security tasks, it is in- 
deed possible to design user interfaces that are less error-prone for 
untrained users. 

To answer the third question, we trained users from the first 
study on security principles, attacks, and tools. We then repeated 
the experiment using unmodified browsers. Our results show that 
education can indeed greatly improve how securely users behave. 
However, security education had significantly less impact than did 
CSCV and had about the same impact as did SPW. 

The rest of this paper is organized as follows. Section [2|shows 
how easily available tools enable eavesdropping and MITM at- 
tacks against Web-based applications. Section [3]summarizes how 
HTTPS and server certificates would theoretically protect Web- 
based applications from such attacks, and discusses why current 
Web browsers allow users to defeat such protection. Section 
analyzes the causes for certificate verification error, distinguishing 
contexts where errors are common and recoverable, from contexts 
where errors are clearly anomalous and suggestive of an attack. 
Sections|5]and|[6|describe CSCV and SPW, respectively. Section[7] 
discusses the design principles underlying CSCV and SPW. Sec- 
tion [8] reviews an alternative approach for security training, and 
proposes a variant suitable for browsers, Staged PKI Client (SP- 
KIC). Sections|9Jand|10] respectively, report and discuss the results 
of user studies performed for evaluating CSCV, SPW, and SPKI. 
Sections[I T]and[I2]discuss related and future work, and Section[13] 
concludes. 


2. EAVESDROPPING AND MAN-IN- 
THE-MIDDLE ATTACKS 


This section explains how easily available tools enable eaves- 
dropping and MITM attacks against Web applications. 

Eavesdropping attacks are enabled by the use of shared media in 
networks such as Ethernet (with hubs) and Wi-Fi. In an eavesdrop- 
ping attack, the attacker configures the respective network interface 
in promiscuous mode. In this mode, the attacker’s computer re- 
ceives any packets sent on the network, including packets destined 
to other nodes. If packets are unencrypted, the attacker can read 
packets’ data, possibly including passwords and other credentials. 
Many easily available applications can be used for eavesdropping, 
including tcpdump and ethereal (8). 

In networks that do not use shared media (e.g., switched Eth- 
ernet) or where packets are encrypted, an attacker may be able to 
use a MITM attack to intercept communication between a client 
and a server. By impersonating the server, the attacker may be able 
to fool the client into connecting with the attacker rather than the 
server. The attacker can then capture the client’s credentials (e.g., 
id and password), and use those credentials to connect to the server 
as the client. The attacker can relay packets between the client and 
the server, making the communication appear normal. However, 
the attacker can read, modify, inject, or drop any packet, even if 
client and server authenticate and encrypt all packets. 

Tools for performing MITM attacks are freely available on the 
Web. For example, arpspoof and dnsspoof can be used when 
client and attacker are connected to the same network [28], as il- 
lustrated in Fig. [I] The first tool, arpspoof, sends to the client 
spoofed ARP packets that associate the attacker’s MAC address 
with a local router’s IP address. Consequently, the client sends to 
the attacker packets that would normally be forwarded through the 


490 


Web 
server 
Internet 
1 2b 
router 
switch 
web |4, ~ | MITM 
client attacker 


Figure 1: Before establishing a connection with a Web server 
(1), the client needs the MAC address of the router that con- 
nects it to the Internet (obtained using the ARP protocol) and 
the IP address of the server (obtained using the DNS proto- 
col). By sending spoofed ARP or DNS responses to the client, 
a man-in-the-middle attacker (MITM) can cause the client to 
send to the MITM packets intended for the server (2a). The 
MITM can then eavesdrop, forge, or modify packets between 
client and server (2b), even if SSL is used. 


router. The second tool, dnsspoof, sends to the client spoofed 
DNS packets that associate the attacker’s IP address with a server’s 
domain name. It causes the client to send to the attacker packets 
that are actually destined to the server. Another tool, webmitm, 
relays intercepted HTTP and HTTPS traffic between client and 
server (28). It usually allows an attacker to capture client creden- 
tials effortlessly, even if communication with the server is SSL- 
secured. 

On Wi-Fi networks, MITM attackers can use another easily 
available tool, airsnarf (25). This tool includes a rogue access 
point utility with built-in DHCP, DNS, and HTTP spoofing. At- 
tackers can cause clients to reassociate to the rogue access point by 
presenting a stronger signal than that of a legitimate access point 
(e.g., by being closer to the client or by using an amplifier or direc- 
tional antenna). Attackers thus intercept all packets between client 
and the server, and can easily capture client credentials. 


3. CONVENTIONAL CERTIFICATE 
VERIFICATION 


This section discusses why, in theory, MITM attacks against 
secure Web servers would not be possible, and why, in practice, 
clients’ Web browsers enable such attacks. 

In principle, MITM attacks would not be possible in secure 
Web communication using HTTPS. HTTPS uses SSL. In SSL, the 
client authenticates the server using public-key cryptography and 
the server’s certificate, as explained in the following. 

A principal (e.g., Web server) can prove its identity to a chal- 
lenger (e.g., client) by encrypting with the principal’s private key 
and a public-key encryption algorithm (e.g., RSA 22) nonces re- 
ceived from the challenger. (A nonce is a cryptographically random 
number that is never reused.) Each private key is known only by 
the respective principal. The private key corresponds to a public 
key that can be publicly known. Any challenger that knows a prin- 
cipal’s public key can use it to decrypt the principal’s response and 
verify the principal’s identity. Challengers typically rely on cer- 
tificates to determine a principal’s public key. Certificates are data 
structures that associate a principal with a public key and are signed 
by a certifying authority (CA). The signature is a secure hash of the 
data structure, encrypted with the CA’s private key. Any party that 
knows a CA’s public key can validate the CA’s signature and thus 


Security Alert 


p 


Information you exchange with this site cannot be viewed or 
changed by others. However, there is a problem with the site's 
security certificate. 


The security certificate was issued by a company you have 
not chosen to trust. View the certificate to determine whether 
you want to trust the certifying authority. 


iv} The security certificate date is valid. 


© The security certificate has a valid name matching the name 
of the page you are trying to view. 


Do you want to proceed? 


|View Certificate | 


Figure 2: Internet Explorer 6.0 dialog when server certificate 
cannot be verified because public key of issuing CA is unknown. 


verify certificates issued by the CA. The public keys of major CAs 
(e.g., Verisign) are embedded in many client applications (e.g., Web 
browsers). 

It is generally believed that it is infeasible for attackers to break 
public-key cryptographic algorithms. Consequently, when a server 
uses HTTPS, it would be infeasible for an attacker to impersonate 
the server, as necessary for a MITM attack. 

However, the current state of Public Key Infrastructure (PKI) 
deployment is such that legitimate Web servers often present to 
clients certificates that clients cannot verify. To accommodate such 
servers, Web browsers typically allow users to override certificate 
verification errors and establish an HTTPS connection even when 
the server has not been authenticated. Fi g.[2|shows a representative 
dialog of the Internet Explorer 6.0 browser in such a situation. 

Unfortunately, the ability to override certificate verification er- 
rors exposes users to the risk of MITM attacks even when HTTPS 
is used. An attacker can easily forge a certificate that associates a 
server with a public key whose corresponding private key the at- 
tacker knows. The attacker cannot get a major CA to sign such a 
certificate (major CAs carefully verify identity out-of-band before 
issuing a certificate). However, the attacker can set up his or her 
own CA that signs the certificate. A client’s browser will be unable 
to verify such a certificate and will likely display a warning such as 
the one shown in Fig.|2| If the user overrides the certificate veri- 
fication error and accepts the attacker’s forged certificate, the user 
will fall prey to a MITM attack, despite SSL protection. 


4. WHY DOES CERTIFICATE 
VERIFICATION FAIL? 


This section analyzes the causes of certificate verification er- 
rors, characterizes cases where such errors are common rather than 
anomalous, and critiques how such errors are presented to users in 
existing Web browsers. 

There are three main causes of user-visible certificate verifica- 
tion errors in Web browsers. First, the browser may not know the 
public key of the CA that issued the server’s certificate. Second, 
the issuer’s or the server’s certificate may be expired. Third, the 
server may have presented a certificate whose common name field 
does not match the server’s fully qualified domain name. (Other 
certificate errors, such as syntax or signature errors, typically cause 
browsers to reject a connection without giving users override abil- 
ity.) 
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The first cause occurs frequently in the case of an internal Web 
server, i.e., a Web server intended for use only by members of the 
organization that owns it (e.g., a university’s students or a com- 
pany’s employees). Major CAs charge a significant annual fee for 
issuing a certificate for each server. Many organizations therefore 
choose to establish their own private CAs, i.e., CAs not certified by 
a major CA, for issuing certificates for internal servers. However, 
the public key of a private CA needs to be installed in client Web 
browsers to avoid verification errors. When the number of users is 
high or client computers are user-owned, this chore often ends up 
being neglected, and certificate verification errors commonly en- 
sue. The existence of private CAs is a de facto situation that does 
not actually conform to existing PKI standards. Therefore, no firm 
guidelines can be found in the literature for how to handle them. 
Existing browsers simply leave the matter on the users’ hands, by 
giving users override ability when such errors happen. 

On the other hand, for public Web servers, i.e., Web servers open 
to the public, e.g., in bona fide financial institutions or e-merchants, 
errors due to the first cause are unexpected and suggestive of a 
MITM attack. 

Certificate verification errors due to the second cause, expira- 
tion, could be caused by something as benign as inattentiveness of 
system administrators, and cannot be easily exploited by MITM at- 
tackers. It appears justified in this case to warn users and give them 
the ability to proceed, as is currently done. 

Errors due to the third cause, name mismatch, can be very se- 
rious. Instead of using a self-signed certificate with a fake iden- 
tity, a MITM attacker can use his or her real identity on a certifi- 
cate from a major CA. Alternatively, a MITM attacker may use a 
major-CA certificate previously stolen from another server (along 
with the respective private key). In MITM attacks using certificates 
issued by major CAs, the certificate’s only flaw may be that it be- 
longs to a common name that differs from that of the Web server 
the user wishes to access. If the domains (last two components of 
the names) are distinct, the possibility of a MITM attack is high, 
and the user should not be allowed to proceed. On the other hand, 
name mismatch at the subdomain level (e.g., s10.acme.com vs. 
s3.acme.com) might be caused by sloppy management or server 
rearrangement at the target organization. It would also be difficult 
for a MITM attacker to get from a major CA a certificate with name 
mismatch only at the subdomain level. It therefore seems reason- 
able in the latter cases to warn users and give them the opportunity 
to proceed, as is currently done. 

Certificate verification errors in existing Web browsers cause the 
browser to identify the cause(s) to the user and allow the user to 
establish a connection despite the error. Fig. [2] shows an example 
from Internet Explorer 6.0. Note that some of the information pre- 
sented to the user is misleading. The alert says that “information 
you exchange with this site cannot be viewed or changed by oth- 
ers,” which is incorrect in the case of a MITM attack. Moreover, 
the alert invites the user to view the certificate to decide whether 
to trust the CA. However, if the browser was unable to verify the 
issuer’s signature on a certificate, viewing the certificate does not 
actually provide any basis for trust, since all the information in the 
failed certificate could be forged by a MITM attacker. Note also 
that the alert does not indicate the severity of the error or provide 
alternatives for overcoming it. 


5. CONTEXT-SENSITIVE CERTIFICATE 
VERIFICATION 


This section describes CSCV, a novel user interface technique 
that applies the previous section’s analysis. CSCV guides users for 


¥ Securit ining | ox 
This browser does not have the 
information it needs to access securely 
the website you requested. 


Do you have the requested site's 
security information (certificate) on 
portable media (for example, CD-R, 
USB, or floppy disk}? 


Ves | 


Figure 3: CSCV accepts private CA certificates on removable 
media. 


¥ Website Certified | an Unknown Authorit Ox 


You need to meet the site's administrator for obtaining and installing 
the site's security information (certificate) on your computer. The 
site provides the following UNVERIFIED contact information: 


Robert Johnson 

Sennott Square Bldg., Room 6503 

210 S. Bouquet St., Pittsburgh, PA, 15260 
Tel: 412-624-7254 

Fax: 412-624-8854 

Workhours: Mon-Fri, SAM-5PM 


You will need to write down and authenticate this contact 
information. Use a PRINTED phone list to verify the administrator's 
phone and fax numbers. If you do not already know the 
administrator, check his or her photo id card when you meet. 


Check Certificate 


Examine Certificate... 


Figure 4: CSCV dialog when user is internal member of the 
organization that owns the accessed server. 


secure handling of certificate verification errors, without override 
mechanisms. 

CSCV thwarts MITM attacks against HTTPS by clarifying the 
relationship between the user and the server whose certificate ver- 
ification failed. As explained in the previous section, internal Web 
servers routinely cause certificate verification errors. To accommo- 
date such servers without giving users override mechanisms, CSCV 
introduces tokens and modifies Web browsers and certificates is- 
sued by private CAs as explained in the following paragraphs. 

CSCV-aware private CAs give to the respective organization’s 
members tokens containing the CA’s self-signed certificate. This 
certificate includes the CA’s public key and is signed with the CA’s 
private key. The tokens are distributed to members securely out- 
of-band on removable media, such as CDR, USB, or floppy disks. 
CSCV-aware Web browsers allow organization members to employ 
these tokens for verifying server certificates issued by the organi- 
zation’s private CA. 

CSCV-aware private CAs also include the CA’s contact informa- 
tion in the issuer alternative name field of server certificates they 
issue [14]. This field typically would otherwise be unused. A 
CA’s contact information normally includes the CA administrator’s 
name, address, telephone and fax numbers, and work hours. CSCV- 
aware Web browsers present this information to users to guide them 
in the recovery from certificate verification errors. 

When certificate verification fails because the public key of the 
certificate’s issuer is unknown, CSCV-aware Web browsers ask 
whether the user has the necessary security information (issuing 
CA’s self-signed certificate) on removable media (e.g., floppy or 
USB disk), as shown in Fig.}] 

If the user does not have the CA’s certificate, CSC V-aware brows- 
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v Website Certified by an Unknown Authority x 
BROWSER WARNING 


This is unusual. Secure websites that are open to the public (such 
as financial institutions, e-merchants, and public web email sites) 
usually provide to the browser verifiable identification, but this site 
does not. 


BE CAREFUL! A hacker may be posing as the site you want to 
access. DO NOT PROVIDE PERSONAL INFORMATION to this 
site before you verify its identity! A hacker could put your 
financial or other private information in danger. 


You need to contact the site's administrator. The site provides the 
following UNVERIFIED contact information: 


Jane Doe 

503 Allen Hall 

127 E. 7th St., New York, NY 10015 
Tel: 212-628-0326 

Fax: 212-628-0325 

Workhours: Mon-Fri, 9AM-5PM 


You will need to write down and authenticate this contact 
information. If you contact the administrator by phone or fax, use 
a PRINTED phone book to verify beforehand his or her number. 
If you meet the administrator or representative physically, check 
his or her photo id card and documents showing that he or she in 
fact is the site's administrator. 


The administrator will likely give you the site's security information 
(certificate) on portable media (for example, floppy disk, CD-R, 
or USB disk). You can then access the site securely. It is most 
secure not to install the certificate, and instead use the portable 
media provided when accessing the respective site. 


Check Certificate 


OK 


Figure 5: CSCV dialog when user is simply a client of the orga- 
nization that owns the accessed server. 


ers ask whether the user is an internal member (e.g., student or em- 
ployee) of the organization that owns the server the user wishes to 
access. If so, the browser displays the CA’s contact information 
and tells the user how to verify the contact information and the ad- 
ministrator, as illustrated in Fig. [4] Users thus learn how to obtain, 
securely and out-of-band, the private CA’s self-signed certificate. 
After installing the certificate, users can authenticate the organiza- 
tion’s servers securely, without resorting to overrides. 

On the other hand, if the user is simply a client of the organiza- 
tion that owns the server, CSC V-aware browsers warn the user that 
the situation is unusual, because secure Web sites that are open to 
the public (such as those of financial institutions and e-merchants) 
typically can be verified by client software (since the respective 
certificates are usually issued by major CAs). The browser also 
explains in plain language, as illustrated in Fig. 5] that a MITM at- 
tack may be happening, and that an attacker could jeopardize the 
user’s financial or other private information. The browser displays 
the CA’s contact information and warns that the latter needs to be 
verified out-of-band. The browser then instructs the user how to 
(try to) obtain the CA’s certificate. The user cannot connect to the 
server without first obtaining the CA’s certificate and securely au- 
thenticating the server. 


6. SPECIFIC PASSWORD WARNINGS 


This section critiques how existing browsers treat transmission 
of unencrypted data, and introduces SPW, a new user-interface 
technique specifically for alerting users against transmitting pass- 
words unencrypted. 

Existing Web browsers can be configured to warn users when 


Internet Explorer 


When you send information to the Internet, it might be 
possible for others to see that information. Do you want 
to continue? 


In the future, do not show this message. 


Figure 6: Internet Explorer 6.0 dialog when the user is about 
to send unencrypted data. 


| Confirm 


2 


DANGER!!! 


This Web site is asking you to send your PASSWORD in an insecure way. Does this password 
protect an account that you wouldn't want strangers to access (e.g., bank, credit card, 


e-merchant, or e-mail account)? 


Figure 7: SPW dialog when user is about to send password 
unencrypted. 


they are about to send unencrypted data. Fig. [6|shows a represen- 
tative example from Internet Explorer 6.0. Because these warnings 
do not discriminate between passwords and other data, they hap- 
pen quite often. Moreover, they do not explain to users the possible 
threats and consequences of sending a password unencrypted. Con- 
sequently, many users disable or ignore such warnings. 

SPW-aware browsers detect when a user is about to transmit a 
password in plaintext (e.g., when accessing an insecure Web server) 
and ask the user whether the password protects an account that 
the user wouldn’t want strangers to access (e.g., bank or email ac- 
count), as shown in Fig. [7] If so, the browser strongly discourages 
the user from continuing. The browser informs that such accounts 
should be accessed securely and explains simply how the user can 
tell whether a site is being accessed securely. The browser also asks 
the user to consider whether the current server is an insecure replica 
of a server that the client normally accesses securely, and explains 
that such a replica could be used in a MITM attack. The browser 
cautions the user that an eavesdropper on the network may be able 
to capture the user’s password and later impersonate the user, with 
possibly significant financial or privacy loss to the user, as shown 
in Fig. |8| Finally, the browser asks whether the user is willing to 
accept all the mentioned risks. 

On the other hand, if the user wouldn’t mind strangers having 
access to the contents of the account, an SPW-aware browser still 
cautions the user that an attacker might be able to capture the user’s 
password and later impersonate the user. The browser then asks 
whether the user wishes to continue. 

SPW-aware browsers can detect in one of two ways that a plain- 
text password is about to be sent. First, when a browser receives 
from a server an HTTP 401 (Unauthorized) response message, the 
browser would normally pop up a window where the user enters his 
or her id and password. If HTTP’s Basic authentication scheme is 
the strongest authentication alternative allowed by the 401 response 
and understood by the browser, the browser would then retry the 
request that caused the 401 response, including the user’s plaintext 
password {10}. Therefore, in such cases, before popping up the au- 
thentication window, SPW-aware browsers execute the SPW dialog 
described in the above paragraphs. Only if the user insists in contin- 
uing, the browser pops up the authentication window and, after user 
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Confirm 


? 


BROWSER ALERT! 
It is recommended that you DO NOT CONTINUE. 


Web pages that contain private password-protected information usually are SSL-secured. The 
address of such pages typically begins with "https://" (instead of "http://"), and the browser 
indicates that the page is secure by showing a CLOSED padlock on the window's margin. 


The page you are accessing, however, is insecure. This insecure page may be a replica of a 
secure page that you normally use. You may be seeing the insecure replica because an 
attacker is intercepting your communication with the Web site. Itis advised that you report this 
problem to the Web site's administrator before accessing the site. 


If you continue, any eavesdropper in the network may be able to capture your password and 
later access your account impersonating you. This could result in significant financial loss to 
you or unwanted disclosure of your private information. 


Are you sure you want to take these risks? 


= 


J 


Figure 8: SPW follow-on dialog when user indicates that pass- 
word protects an important account. 


input, sends the id and password unencrypted to the server. The 
browser also associates the id and password with the realm speci- 
fied in the 401 message, and caches them for the current browsing 
session. The cache makes it unnecessary for the user to enter the 
same information again when the browser receives a new 401 mes- 
sage in the same realm. 

Second, the user may enter data into an HTML form with an in- 
put element of type password. (Form input elements of this type 
echo as asterisks the characters the user types.) When the user sub- 
mits such a form, an SPW-aware browser checks the action associ- 
ated with the form. If the action specifies a URL that uses HTTP 
rather than HTTPS, the browser executes the SPW dialog and starts 
the action only if the user insists in continuing. 

Note that, unlike CSCV, SPW allows user overrides. This is be- 
cause some legitimate password-protected Web servers, e.g., for 
Web-based email access, do not use HTTPS. In cases where pass- 
words entered into an HTML form are processed by Javascript 
in the user’s browser, servers may implement challenge/response 
schemes that actually protect user passwords reasonably well from 
eavesdropping and replay attacks (10). Although such schemes do 
not protect the accessed content from eavesdropping, a user over- 
ride in such cases may be justified. 


7. JUST-IN-TIME INSTRUCTION 


This section discusses the design principles underlying CSCV 
and SPW. 

Currently, most personal computer software uses some form of 
just-in-time instruction (JIT!) (6) to handle security errors. JITI 
gives the user information that the software writer deems important 
for the user to make a decision at a critical point of the processing 
of a task or error. At such a point, the software presents to the user 
a usually short message on a pop-up window. Interested users can 
learn more by following hyperlinks embedded in the message, or 
by searching the software’s help facility. Users usually can also 
simply dismiss the message. 

JITI can have many shortcomings. First, JITI messages often use 
concepts that nonspecialists do not understand. Second, JITI mes- 
sages may not fully or correctly disclose possible consequences of 
users’ decisions (see, e.g., Fig. [2). Third, JITI messages usu- 
ally do not tell users how they might overcome security errors: the 
software simply asks user permission to continue a task, despite 
lack of security. Fourth, because of the above problems, when JITI 
presents a security warning, time-pressed nonspecialists often sim- 
ply click “continue,” without fully investigating and understanding 
the risks involved. Over time, this behavior can become a hard-to- 
kick habit of dismissing JITI warnings. 


We call warn and continue (WC) the class of user interfaces that 
implement JITI with all four of the above shortcomings. The cur- 
rent Internet Explorer (IE) 6.0 is a good example in this class. 

CSCV is representative of another JITI class, which we call guid- 
ance without override (GWO). User interfaces in this class disclose 
risks in plain language and fully. They tell the user how to elimi- 
nate the conditions that caused a security warning, and do not allow 
the user to proceed without doing so. The user interface tailors the 
instructions for the specific situation the user is facing. In order 
to do so, the user interface may need to gather information from 
the user, from the server the user wishes to contact, or from other 
computers. 

SPW belongs to yet another JITI class, which we call guidance 
with override (G+O). User interfaces in this class also disclose risks 
in plain language and fully, and may give users tailored instructions 
for eliminating the conditions that caused a warning. They may also 
diagnose the user’s situation by inquiring the user. However, unlike 
GWO, G+O lets the user proceed without eliminating conditions 
that caused a warning. Because it does not need information or 
accommodations from servers that users access, G+O may be more 
broadly applicable than is GWO. On the other hand, G+O is also 
more vulnerable to user errors than is GWO. 


8. WELL-IN-ADVANCE INSTRUCTION 


This section discusses an alternative approach for achieving se- 
curity via user training. In the next section, we experimentally com- 
pare an instance of this approach, SPKIC, with CSCV and SPW. 

Well-in-Advance Instruction (WIAI) is a user interface design 
principle recently proposed by Whitten for replacement of 
JITI. According to Whitten, JITI’s main problem is that it leaves 
instruction to the last minute. Users are then typically focused on a 
task for which security is not a primary goal. Consequently, users 
are unlikely to try to learn security concepts at that time. Whitten 
argues that user interfaces should teach necessary security concepts 
before the user actually needs them. 

A good way to implement WIAI, says Whitten, is via safe stag- 
ing (30). In safe staging, software varies the user interface accord- 
ing to the user’s stage. Each stage should enable only functions 
and data that the user knows how to manipulate safely (e.g., the 
whole functionality on dummy data, or only safe functions on the 
user’s own data). During each stage, software encourages the user 
to learn the concepts necessary to progress to the next stage. In 
hard staging, the software allows progression only when the user 
has demonstrated mastery of the relevant content. On the other 
hand, in soft staging there is no such enforcement. 

Whitten implemented an email agent, Lime, that uses soft stag- 
ing to teach users how to use public-key certificates under PGP’s 
Web-of-Trust model. Such certificates are necessary for email au- 
thentication and confidentiality. In Lime’s first stage, users are as- 
sumed to know how to trade keys on removable media. In this 
stage, the user can send secure email only to people the user has 
already personally met and traded keys with. In the second stage, 
the user may send email with the user’s self-signed certificate and 
a few personal remarks that the recipient can use to identify the 
sender’s identity. In this stage, the user can send fairly secure email 
to any acquaintances, even those with whom the user has not traded 
certificates; however, the data must not be very important, since se- 
curity is not high. In the third stage, the user can send secure email 
to anybody certified by a person the user has previously met and 
traded keys with. In this stage, there are no functional or data re- 
strictions. We call this methodology staged web-of-trust (SWOT). 

SWOT is not directly applicable to browsers, among other rea- 
sons because SSL certificates are based on PKI, not Web of Trust. 
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We therefore propose another similar soft staging methodology for 
teaching users how to deal with PKI-based certificates and unpro- 
tected passwords in Internet Explorer. We call this methodology 
staged PKI client (SPKIC). In SPKIC’s first stage, users learn not 
to accept unverified certificates and to avoid sending unprotected 
passwords. Users also learn how to obtain the certificate of a 
private CA. (Similar instruction is not provided in Lime because 
the notion of meeting an intended email correspondent and trading 
keys is considered intuitive enough. However, in the case of a Web 
browser, it is not obvious to the user who, where, and when to meet 
in order to get a CA’s certificate securely.) SPKIC-aware software 
enforce the mentioned restrictions so as to make this stage safe. In 
SPKIC’s second stage, users learn about MITM and eavesdropping 
attacks and how to use OpenSSL to set up a private CA and issue 
bona-fide and bogus certificates. Users also learn how to modify 
an SSL client program so that it uses the OpenSSL library to verify 
certificates received from servers. (Whereas the first stage taught 
the “how” of dealing with certificates and passwords, the second 
stage focuses on the “why.” SWOT’s second stage achieves a sim- 
ilar goal using a simpler “social authentication” paradigm. Social 
authentication is well-suited to email, but unfortunately not appli- 
cable to Web browsing.) This stage is safe because only dummy 
data is used. Finally, in SPKIC’s third stage, users employ unmod- 
ified Internet Explorer to visit a variety of sites. In this stage, there 
are no restrictions because the user is assumed to have the knowl- 
edge necessary to behave safely. 


9. EXPERIMENTAL RESULTS 


This section reports the results of three user studies performed to 
evaluate CSCV, SPW, and SPKIC. 

The users in the first two studies were 17 male Computer Science 
undergraduates in their senior year. The first user study provides a 
baseline for security before any browser modifications or training 
about certificates and MITM and eavesdropping attacks. In it, users 
employed an unmodified Internet Explorer 6.0 browser. The second 
user study was performed back-to-back with the first study. In it, 
the same users employed a modified version of the Mozilla Fire- 
bird 0.6.1 browser with CSCV and SPW. No feedback or further 
information was given to students between the first two studies. 

The second user study served also as the first stage of the third 
study. Twelve of the original seventeen students then received train- 
ing on certificates and MITM and eavesdropping attacks, according 
to the SPKIC methodology described in the previous section. This 
stage lasted approximately one month. Students then progressed to 
SPKIC’s third stage, where they again used the unmodified Internet 
Explorer 6.0 browser. 

In each user study, we asked students to visit three fictional 
but realistic Web sites where students were assigned password- 
protected accounts. The first site is maintained by the students’ uni- 
versity. It allows students to monitor the respective reward points. 
Students earn these points by doing well in exams, independent 
studies, or service to the university. The second site is maintained 
by a remote e-merchant that is not affiliated with the university and 
where students can spend their reward points, e.g., to buy books, 
CDs, or spring break vacations. The third site provides access 
to users’ Web email accounts. We asked students to verify their 
balance on the first site, spend some of it by ordering something 
from the second site, and get an order confirmation message on the 
third site. We asked students to consider that reward points have 
real monetary value and to exercise regular care while browsing. 
We pointed out that a student could decide not to visit a site if 
he deemed the visit too risky. We asked students to “think aloud” 
while browsing and describe what they were doing and why. 


Study 1: 17 users Study 2: 17 users Study 3: 12 users 
Browser Unmodified Internet Explorer || Modified Mozilla Firebird || Unmodified Internet Explorer 
UI methodology Warn and continue (WC) CSCV SPW Staged PKI client (SPKIC) 
Site type UC/M | UC/C NC UC/M | UC/C NC UC/M | UC/C NC 
Score 0 17 16 11 0 0 2 6 5 2 
frequency | 50 0 - - 1 - - 2 - - 
100 0 1 6 16 17 15 4 7 10 
Average Score 0 6 35 97 100 88 42 58 83 


Table 1: Users’ security scores when accessing sites of various types. CSCV and SPW greatly improved security scores of untrained 
users on sites of all types. The effect of security training with SPKIC was smaller than that of CSCV, but similar to that of SPW. Please 
consult Table|2]for statistical significance of these results. On Tables|1]and [2] “UY” stands for user interface, ““‘UC/M” represents site 
with unverified certificate and belonging to organization user is member of, “UC/C” is site with unverified certificate and belonging 
to organization user is simply a client of, and “NC” is site without certificate (no SSL). 


All studies took place in a laboratory that the students had al- 
ready been using for course assignments. By the student’s com- 
puter, we provided a telephone and phone directories (phone com- 
pany’s local white and yellow pages and the university’s directory). 
We configured the first two sites with HTTPS and server certifi- 
cates issued by private CAs whose public key was unknown by 
client software. The first site’s CA contact information was that of 
a real person listed on the university’s directory, with office on a 
different floor in the same building as the laboratory. The second 
site’s certificate was bogus. We configured the third site with HTTP 
only (no SSL). 

We scored how securely users accessed the sites as follows. If a 
user accessed a site despite lack of security, the user got 0 points. In 
the first site, if a user simply did not visit the site insecurely, the user 
got 50 points. If the user also correctly obtained and installed the 
issuing CA’s certificate and thus accessed the server after properly 
authenticating the server, the user got 100 points. Lack of security 
in the second and third sites could not be corrected. Thus, users 
who simply did not visit each site insecurely got 100 points. The 
students’ security scores are represented on Table[I] 

In the first study, users’ think-aloud comments indicated that 
they were quite familiar with Internet Explorer’s dialog for unver- 
ified server certificates (Fig. B). A typical comment was “um, an- 
other of those pop-ups.” Most users dismissed the warning with a 
comment like “I always just click yes when I see these pop-ups.” 
Other users viewed the servers’ certificates and, with only one ex- 
ception, accepted them. A typical comment was, “looks legit to 
me.” The one exception was a user who expressed concern about 
the e-merchant’s certificate not being signed by a major CA. How- 
ever, the same user accepted the certificate of his school’s server, 
which also was unverified and could have been forged. Interest- 
ingly, about a third of the users found that sending their password 
unencrypted was too risky and decided not to visit the third site. 

In the second study, users initially displayed some uneasiness 
about the modified browser’s dialogs (Figs. #ļand [5). The dialogs 
ask the user to contact and verify the site’s administrator. A typical 
user reaction was, “Am I really supposed to do this?” Many users 
tried to start again and give different answers so as to be able to ac- 
cess the site despite the certificate verification error. However, the 
modified browser does not provide any way to override the verifi- 
cation process, and users eventually followed the browser’s direc- 
tions for obtaining the private CA’s certificate. The one exception 
was a user who decided not to visit the first site at all. The modi- 
fied browser was very successful also at discouraging unencrypted 
transmission of passwords. A typical comment to the dialog in 
Fig. [8]was, “T guess I better not access this site.” 

In the third study, about half of the users were deliberate, exam- 
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ined the certificates, commented on the possibility of forgery and 
man-in-the-middle attack, and decided not to visit the sites. How- 
ever, the other users continued to accept unverified server certifi- 
cates, with comments similar to those in the first study. Most users 
decided not to send unencrypted passwords, with comments like 
“it’s too risky.” 

Table [2] shows the statistical evaluation of the hypotheses tested 
by the user studies. Roughly, p-values estimate the likelihood that 
observed differences (shown in Table[Ip are due to chance. Exper- 
imental evidence in favor of a hypothesis is by usual convention 
considered highly significant if p-value < 1%, significant if 1% < 
p-value < 5%, or not significant otherwise. We used the Wilcoxon 
signed-ranks (nonparametric) test to determine p-values. We con- 
sidered Wilcoxon more suitable than a parametric test because the 
dependent variable in our experiments (i.e., security score) had only 
two or three possible values. Parametric tests, on the contrary, as- 
sume that the dependent variable has a fairly large number of pos- 
sible values. We used Hommel’s procedure to correct p-values for 
the fact that the user studies test multiple hypotheses {13} 24}. Be- 
cause the procedure discarded as not significant only one of the 
hypotheses, the correction factor was 1. 


10. DISCUSSION 


Results for the first user study suggest that the actual security 
of existing browsers is appalling, when the “human in the loop” 
is considered. Because most users dismiss certificate verification 
error messages, SSL provides little real protection against MITM 
attacks. Users actually behaved less insecurely when interacting 
with the site that was not SSL-secured. 

Results for the second study suggest that CSCV and SPW greatly 
improve browser security and are easy to use even without training. 
CSCV is used on the first two sites and resulted in higher scores 
than did SPW, which is used on the third site. All but one of the 
users were able to follow CSCV’s online guidance for securely ob- 
taining and installing the certificate of the private CA maintained 
by the user’s organization. These users then accessed the organiza- 
tion’s server securely. Observation of the users’ behavior indicates 
that the absence of override ability was essential for achieving a 
high level of security and had only modest impact on usability. 
None of the users accessed the e-merchant’s site insecurely, and 
most users refused insecure access to Web email. 

Caution is needed in interpreting results of user studies per- 
formed in a laboratory, such as the ones reported in this paper. 
When users receive explicit instructions to perform certain tasks 
and know they are being observed, they may be more likely to com- 
plete the tasks than they would be in real life. Such a bias would 
tend to increase the security score for Study 2’s first site, where the 


Hypothesis p-value Significance 

CSCV provides higher security scores than does IE’s WC in UC/M sites | < 0.0005 | highly significant 
CSCV provides higher security scores than does IE’s WC in UC/C sites < 0.0005 | highly significant 
SPW provides higher security scores than does IE’s WC in NC sites 0.003 highly significant 
SPKIC provides higher security scores than does IE’s WC in UC/M sites 0.023 significant 
SPKIC provides higher security scores than does IE’s WC in UC/C sites 0.014 significant 
SPKIC provides higher security scores than does IE’s WC in NC sites 0.008 highly significant 
CSCV provides higher security scores than does SPKIC in UC/M sites 0.011 significant 
CSCV provides higher security scores than does SPKIC in UC/C sites 0.025 significant 
SPW provides higher security scores than does SPKIC in NC sites not signif. not significant 


Table 2: Statistical evaluation of the hypotheses tested by the user studies. P-values were calculated using Wilcoxon’s signed-ranks 
test, with familywise error rate controlled by Hommel’s procedure. 


user needs to go meet the CA contact for the score to reach 100. 
For the other sites and studies, the same bias would tend to depress 
security scores. Review of observations of user behavior and think- 
aloud comments suggests that this bias was significant for Study 
2’s first site, but not for the other sites and studies. We did not 
control this effect because, in real life, an organization’s members 
usually do have compulsory reasons to access the organization’s se- 
cure Web sites, and therefore would need to get the organization’s 
private CA’s certificate. For example, a student may need secure 
Web access to a university’s servers to register for classes, reserve 
a book from the library, or check her grades; an employee may need 
secure Web access to check his email, get a parking permit, or file 
expense reports. 

If a private CA issues certificates only for servers whose access 
is optional, it is possible that a significant number of users may not 
bother to get the CA’s certificate or access the protected informa- 
tion. The likelihood of task completion is affected by the difficulty 
of such completion. In our experiments, the bona-fide CA contact 
was in the same building as the users. In the case of an organization 
distributed across several buildings or campuses, it may be advis- 
able to maintain multiple certificates for secure servers. Servers 
can then use for each client a certificate with CA contact next to 
the client’s location. If this is not possible, or users’ motivation to 
access an organization’s secure sites is low, system administrators 
need to distribute the private CA’s certificate to users proactively 
(e.g., on removable media, as suggested in Section B}. rather than 
wait for users to come looking for them. Alternatively, system ad- 
ministrators may prefer to use a major CA in such applications. 

Results of all three user studies could have been biased by users’ 
age, gender, education, and ability. Additionally, results of the 
third user study could have been biased by the instructor’s abil- 
ity and particular methodology (SPKIC) used. It is unlikely that 
SPKIC could be used with a much more diverse user population, 
given the inclusion of programming tasks in SPKIC’s second stage. 
In some sense, SPKIC’s second stage may appear to approach an 
upper bound on the training that could realistically be achieved. 
However, interventions such as drill exercises using the actual final 
browser might actually be more effective. It is also possible that 
SPKIC would be more successful if the final browser were not one 
which the users had already acquired bad habits with, or if the final 
browser’s user interface were modified, e.g., to end within the G+O 
class instead of the WC class. 

Given the differences in methodology, results of our third user 
study cannot be readily compared with those of Lime. Lime’s 
study had a more diverse user population, with ages ranging be- 
tween 18 and 41 years, education ranging from some college to 
Master’s degree, and various areas of expertise, including com- 
puter science, engineering, psychology, and history (31). Users 
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who had used PGP before were excluded from Lime’s study. Ad- 
ditionally, Lime incorporated carefully designed messages and vi- 
sual metaphors that would place its user interface within the G+O 
class. Nonetheless, at the beginning of Lime’s third stage, 8 out of 
10 users accepted unverified certificates. Users then received com- 
plaints about their own unverified certificates — a common event 
in PGP email that has no parallel in Web browsing, where clients 
typically do not have their own certificates. When then presented a 
correspondent’s alleged new public key, 2 out of 9 users accepted 
the certificate without verification. Thus, at the end of Lime’s last 
stage, a significant fraction of users were still vulnerable to MITM 
attacks. Although SWOT and SPKIC are quite different, neither 
staged methodology seems to achieve sufficiently usable security 
to deter the kinds of attacks we’re concerned with in this paper. We 
suspect that, given human nature, the same is true of any user in- 
terface that gives users discretion to proceed without fixing serious 
security problems, such as unverified certificates (e.g., within JITI, 
any user interface outside of GWO). 


11. RELATED WORK 


Captive portals are Web servers used to authenticate users in 
many Wi-Fi hotspots. Although captive portals are SSL-secured, 
they are vulnerable to session hijacking, freeloading (36). and 
MITM attacks. In [36], we demonstrated new defenses against 
hijacking and freeloading that are easy to use because they are 
transparent to users and interoperate readily. In (37). we showed 
how such defenses can be integrated with new Wi-Fi security pro- 
tocols and billing schemes. This paper’s techniques complement 
that work by helping thwart MITM attacks in such environments. 

Most previous work on security ignores human factors. The lat- 
ter often make computer systems in practice insecure. An increas- 
ing awareness of this problem is turning user interfaces into an im- 
portant area in the security research agenda (3](39][2}[26] [23][27). 

Our results are consistent with those of Whitten and Tygar’s sem- 
inal work on the usability of public-key cryptography for email se- 
curity . Public-key cryptography is difficult for users to un- 
derstand and use. Our results for CSCV suggest that, at least in 
applications of public-key cryptography to server authentication, it 
is possible to make user interfaces less error-prone. 

Several systems secure the online retrieval of certificates by 
confirming a certificate’s fingerprint out-of-band (e.g., by print- 
ing on the subject’s personally delivered business card or by tele- 
phone) (21). Introduction of such a method in CSCV could make 
CSCV easier to use, since it would allow users to get the private 
CA’s certificate online and confirm it by telephone, instead of per- 
sonally meeting the CA’s contact. User tests need to be performed, 
however, to verify such a method’s effectiveness against MITM at- 
tackers. 


Identity-based cryptographic algorithms do not require certifi- 
cates. They use as a party’s public key the party’s identifier (e.g., 
fully qualified domain name or IP address). Such algorithms are 
actively being researched and, if they prove to be practical, 
they could provide an alternate solution for making Web browsers 
more usably secure. 

Ackerman and Cranor have proposed critics that help Web brows- 
er users understand and negotiate privacy issues that may be in- 
volved in visiting certain sites (i). Interestingly, critics automat- 
ically collect from trusted sites the latest intelligence on privacy 
threats. We believe that similar self-update capability may be able 
to significantly enhance the robustness and effectiveness of G+O 
interfaces, including SPW. 

Ye and Smith have shown that an attacker can use Javascript to 
spoof a browser’s usual signs that a site is being accessed securely 
(“https” on the address line and closed padlock) (38). They propose 
“trusted path” mechanisms that allow users to detect such spoofing. 
However, our results show that, clear warnings notwithstanding, 
users often access sites insecurely if given the chance. We thus 
believe that CSCV and SPW would significantly further improve 
the usable security of browsers with trusted paths. 

Whitten’s safe staging can be interpreted as an amplification and 
application to security of two previous techniques: software train- 
ing wheels and tutoring scaffolding . Training wheels dis- 
able in an initial stage functions that users tend to have problems 
with, whereas scaffolding gives novices help and examples that are 
gradually withdrawn as users become more proficient. 

A large-scale study involving security training by Yan et al. at 
Cambridge University [33] obtained results similar to our own. 
That study attempted to determine the effectiveness of various in- 
terventions for reducing password vulnerability to dictionary at- 
tacks. The study found that teaching students how to choose more 
secure passwords can significantly reduce the incidence of vulner- 
able passwords. However, that incidence was still unsatisfactorily 
high (at least 10%) after the studied interventions. Yan et al. con- 
jecture that to achieve acceptable compliance, computer systems 
need to proactively prevent users from choosing weak passwords, 
rather than simply rely on training. 


12. FUTURE WORK 


There are many other protocols, besides HTTPS, where clients 
use public key cryptography to authenticate servers. We are cur- 
rently investigating how to adapt CSCV to improve the security 
of the SSH (Secure Shell) and the usability of the PEAP 
(Protected Extensible Authentication Protocol) user interfaces. 
Current SSH clients keep the public key of known hosts in a file. 
When a server presents an unknown or different public key to a 
client, the client software asks if the user wants to accept the key 
just for this session or all future sessions. This paper’s results sug- 
gest that such an interface can effectively defeat SSH’s security 
(note that MITM attack tools against SSH, such as sshmitm, are 
easily available on the Web (28}). PEAP is used in the new native 
Wi-Fi security scheme, WPA2/802.11i [32] . Current PEAP clients 
(e.g., Microsoft Windows XP’s and Linux’s xsupplicant (19) sim- 
ply refuse to connect to a network if the latter’s authentication 
server’s certificate cannot be verified. Such a design is secure, but 
difficult to use. We expect CSCV’s guidance to improve PEAP us- 
ability significantly. 

It would be interesting to investigate to what extent habituation 
may decrease the effectiveness of SPW (or, in general, any G+O 
interface): After seeing the same warning many times, users may 
pay less attention to it. On the other hand, CSCV (and, in general, 
any GWO interface) does not allow user override, and is thus not 
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vulnerable to habituation. CSCV’s usability may actually improve 
as users become familiar with CSCV’s guidance. 


13. CONCLUSIONS 


Web browsers need to handle many security errors. The current 
practice favors delegating decisions to the user via pop-up windows 
that allow users to override security failures. Our results suggest 
that the current practice effectively defeats Web security. 

Tools that can be freely downloaded from the Internet enable 
even novice hackers to perpetrate MITM attacks that cause signif- 
icant loss to victims. Existing Web security mechanisms, such as 
server certificates and SSL, in theory protect users from such at- 
tacks. However, most users do not check or understand servers’ 
certificates or ignore warnings that a certificate cannot be verified. 
Consequently, existing Web mechanisms provide little real secu- 
rity to most users. We proposed CSCV, a novel user interface tech- 
nique for handling certificate verification errors. CSCV asks the 
user questions and receives from the server information that enable 
the browser to discriminate the context in which a certificate ver- 
ification error occurs. Considering this context, the browser then 
guides the user in handling and possibly overcoming the error. We 
also proposed SPW for warning users about the specific threats and 
risks involved when they are about to send unencrypted passwords. 
We performed user studies to evaluate CSCV and SPW. Our results 
suggest that CSCV and SPW greatly improve the security of Web 
browsers and are easy to use even by untrained users. 
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